A common software development and/or release task is to determine what portions of the software code have been or have not been executed when the code is run with various test data sets. This measurement is generally termed “code coverage.” Typically, the program being measured is run many times, each time with a different test data set, and the results are combined into a single report as an “aggregated code coverage.” Many companies have a release criteria that a percentage (e.g., 90%, 95%, 98%, etc.) of the code has been exercised by the test data sets which serves as one measurement of quality. This is because the higher the percentage of the code coverage, the higher the confidence that the program has been thoroughly tested and will execute correctly.
There are many approaches to measuring the code coverage of a single run, but they usually suffer the same performance disadvantage especially when aggregated code coverage is done. In those approaches, the code coverage is measured independently and redundantly for each test run. After each run, or, in some cases, after all runs, the results are combined and duplicate code coverage information is discarded. Since most test runs exercise much of the same code with only a little variation from run to run, the performance overhead to capture all of the commonly-executed code coverage information is wasted. Even within a single run, the same code may be repeatedly executed thousands of times. Any added instructions to measure code coverage will likewise be executed thousands of times. After the first execution, subsequent executions are wasted effort.
There are many “types” of code coverage measurements: function, basic block, statement, condition, etc. However, the most common and useful measurement is basic block. A basic block of programming code is a sequence of instructions in which there is only one entrance (the first instruction) and one exit (the last instruction). If one instruction in a basic block is executed, then all instructions in that basic block will be executed. For example, in the statement: if (condition) then {A} else {B}, each of A and B is a basic block. Additional examples are the code in a loop, straight-line code with no control changes, etc.